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ABSTRACT 



This paper describes a methodology for helping institutional 
research assess its roles and its effectiveness in supporting decision-making 
at all institutional levels. The paper discusses how to meet this goal by 
planning, doing, checking, and acting. Planning requires the understanding of 
the roles of the data custodian, the broker, and the information user. Doing 
involves performing these roles through five sequential steps. Checking 
requires understanding and overcoming barriers to effectiveness. Acting 
involves using strategies to integrate and interpret information in order to 
enhance its value to the institution. Individual sections of the paper define 
the term institutional research, establish the framework of the model, and 
discuss management of the various institutional research functions. 
Recommendations and suggestions for overcoming barriers are discussed. A 
table listing barriers to effective institutional research is appended. 
(Contains 15 references.) (CH) 
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ABSTRACT 

The mission of institutional research is to conduct analyses and produce information that 
supports the development of effective institutions and organizations in higher education. The 
institutional researcher must develop a methodology that results in systematic processes, which 
overcomes organizational barriers, and that adds value to the institution.. Effectively meeting 
this goal requires planning, doing, checking, and acting. Planning requires understanding the 
roles of the data custodian, broker, and information user. Doing involves performing these roles 
through five, sequential functional steps. Checking requires understanding and overcoming the 
barriers to effectiveness that surface within each of the five steps. Acting involves using 
strategies to integrate and interpret information in order to enhance its value to the institution. 
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EFFECTIVE INSTITUTIONAL RESEARCH: 
OVERCOMING THE BARRIERS 

The mission of institutional research is to conduct analyses and produce information that 
supports the development of effective institutions in higher education. Institutional research is a 
form of applied research. The initiation of a research project of this type is nearly always 
stimulated by the desire to find an answer to some practical problem. The researcher must be 
objective, detached, thorough, and systematic as any other scholar. Furthermore, problems 
addressed must be well defined, methods must be appropriate, analyses must be logical, and 
conclusions must be unbiased by pre-conceptions, just as those identified for use in basic 
research. The institutional researcher must adhere to standards of excellence in research while 
respecting the role of higher education philosophy in acting as the catalyst for its efforts (Saupe 
and Montgomery, 1970). 

Institutional research supports decision-making at three levels in the institution - 
strategic, managerial, and operational. To accomplish this goal, it must identify a methodology 
that will result in reliable and valid conclusions about the applied problems being investigated. 
The methodology must enable institutional researchers to 1) define the problems; 2) select the 
proper methods of analysis; 3) avoid bias; and, 4) keep the institution focused on its core 
business - higher education. 

This paper describes a methodology for helping institutional research assess its roles and 
its effectiveness in supporting decision-making at all levels of the institution. Recommendations 
and suggestions for overcoming the barriers to success are discussed. 
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INSTITUTIONAL RESEARCH DEFINED 

As a functional area, institutional research serves the organization through critical 
appraisal and careful investigation of processes and progress (Suslow, 1972). Appraisal should 
be systematic, factual, concerned with commonly accepted values in higher education, and more 
or less continuous. In other words, an effective institutional research function, through the 
proper analysis of good data, provides the institution an opportunity to engage in continuous self- 
study to improve institutional effectiveness. 

Institutional research should also support processes that help solve problems shown to be 
important to key individuals in an organization. The processes should provide for timely 
responses. It must be able to employ data in the study that give an accurate representation of the 
problem while educating their customers and end users in correct interpretation of products. To 
accomplish these goals, institutional research must actively engage in learning about customer 
needs and must provide for both the production and delivery of information required to support 
broader institutional activities. 

At the core of understanding what contributes to development of an effective institutional 
research function is the issue of configuration or structure. Institutions of higher education are 
sometimes called Professional Bureaucracies, referring to a system that relies on the skills and 
knowledge of operating professionals in order to function (Mintzberg, 1979). Work of these 
professionals (faculty) is generally coordinated through standards originating largely outside the 
institution in associations formed in cooperation with other colleagues from the profession. The 
staff needed to support the professionals in conducting the institution’s mission is relatively 
large. The staff provides a range of activities such as personnel, printing, computer facilities. 
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and libraries. There is also a strategic apex of the institution comprised of the president and key 
executives. 

The institutional research function fits yet into another area of the institution known as 
the technostructure comprised of analysts who plan and who support the processes that control 
the work of others. This area includes such activities as planning, budgeting, and outcomes 
assessment. Though small in size, the technostructure has gained in importance as the need has 
increased to respond to outside pressures, e.g., state legislatures, college ratings. 

The location of the institutional research function within the technostructure, rather than 
in the strategic apex or staff, is important for a number of reasons. First, those who do IR are 
close enough to the strategic center of gravity of the institution to be knowledgeable of important 
issues and problems. Second, they should be sufficiently outside the administrative staff process 
to maintain objectivity, credibility, and focus. Although institutional research may not 
continuously participate in policy formulation, it must assist policy formulation by specifying 
what is feasible and how effective various strategies are likely to be. The function should also 
wear a third hat by serving as a moderator between technical individuals, academic scholars 
(especially those involved in governance), and administrators responsible for management 
decisions (Sheehan, 1985). 

The effective institutional research function serves institutions that are “concerned with 
their purposes and values as well as their efficiencies and productivity.” (Suslow, 1972) 
Therefore, the institutional research function must engage in activities which bring it into contact 
with the professorate who engage in activities, which help shape the culture of the institution. 
Institutional researchers should understand, and where appropriate, participate in the major 
activities of the faculty to include instruction, research, service and governance. Given the 
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complex relationships between institutional research and other parts of the professional 
bureaucracy, institutional research must seek to identify models that serve as a framework within 
which its purpose and contribution to the value of the organization can be assessed and 
documented. 

THE FRAMEWORK 

The basic model recommended in this paper is the Information Support Circle shown in 
Figure 1 . It is made up of three roles and five functions. The Custodian obtains and stores the 
data, creating an asset for the institution. Data from the Custodian are restructured and 
integrated (converted to information) by a Broker, who then provides the information to the 
Manager for the purpose of supporting decision-making in the institution. The five functions in 
the Information Support Circle represent tasks associated with institutional research. Data are 
identified, captured, stored, restructured, analyzed, and integrated. The resulting information is 
then delivered or communicated in a form that can be integrated into the knowledge base of the 
manager. The result is increased intelligence for the manager, answering some questions and 
raising the next set of questions; and the cycle begins again. The effectiveness of the institutional 
research function thus depends on its involvement in, and integration of, the three roles with the 
five functional areas. The Shewart Cycle of Plan, Do, Check, and Act shown in Figure 2 
provides an overall framework for the paper and is used to explore how to improve the 
performance of institutional research. 
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PLAN, DO, CHECK, ACT: MANAGING THE INSTITUTIONAL RESEARCH 
FUNCTION 

As noted earlier, institutional research is a form of applied research. Though this fact 
presents tremendous opportunities for institutional researchers, failure to properly manage the 
function may create barriers to effectiveness that will deny institutional research its rightful place 
among other functions within the organization. Implementation of a strategy for self-study using 
the Shewart Cycle of Plan, Do, Check, and Act can help institutional research avoid this trap and 
enable it to provide a value-adding function. 

Plan - Three Roles and the Institutional Researcher: 

Planning how to best do IR involves the identification of ways to best coordinate three 
roles (Custodian, Broker, and Manager) in a manner that supports the strategic, managerial, and 
operational needs of the institution. It places major emphasis on recognizing data needed to 
assess the external and internal environments of the institution and on information support 
processes needed to maintain the structural decision-making integrity of the institution. Since 
processes and structures change over time, planning is never complete in a day-to-day sense. 

The information support cycle as defined by the Information Support Circle never ends. 

The nature of the institution and the nature of our work drive planning. Both the 
development of our data and the use of our function add most value when they are integrated 
with planning. Our products take time to develop so we must anticipate the needs they meet. The 
use of our products can help the institution anticipate problems and to initiate solutions given the 
time is available. 
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As knowledge workers, the focal points that drive institutional research processes are 
data, information, and intelligence. Data are the measures of events and facts that represent the 
basis of our observations. They tend to be numerical and objective - How many students? How 
many faculty? How many computers are needed and how much will they cost? At the same 
time, data can be qualitative and subjective - What do students perceive is necessary to get an 
“A”? What value do faculty place on tenure? Qualitative data can generate information needed 
by decision-makers but, at the same time, are subject to different interpretations across 
researchers. However, the degree of reliability with respect to qualitative data may be better than 
the reliability of poorly defined categories on which quantitative counts are made. 

Information is derived from understanding and interpreting the context of the data. 

When the number of students are identified, what does this mean in terms of the revenues they 
generate and the courses they take? What are the reasons for the views and concerns of the 
students or the faculty concerning some relevant issue? What triggered specific events? 
Answering such questions requires that the data be transformed in order for the researcher to 
identify causative and associative factors. In other words, the data must be analyzed and 
restructured in order for the data to become information useful to the end user. 



Intelligence refers to the interpretation of our information as it is generalized to the 
situation. It is the process by which events and facts are diffused and integrated into the 
knowledge base of the members of the community. Development and use of models that 
facilitate interpretation and understanding of cause-effect relationships in a way that meets the 
needs of the end user is critical. Given the analysis and test results, what should be the impact 
of increasing tuition? Given the faculty perspectives, will modification of a specific policy cause 
unrest among faculty? How should priorities be ranked when implementing new programs? 
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Interpretation of the information provides input that can be used to test reality and to evaluate the 
desirability of a decision. 

Institutional research has an opportunity to add value to the institution by putting into 
Place planning systems to effectively manage data, information, and its interpretation. When 
data are needed, institutional research must function as, and with, the Custodian of that data. 
When information is needed, institutional research must function as the Broker of that 
information. When the facts are being intetpreted to increase organizational intelligence to better 
understand a situation, institutional research functions as, and with, the Manager who needs 
those interpretations. The three roles are explored in more detail in the following sections. 

The Custodian. The Custodian acts as a source for internal and external data. With the 
advent of the data warehouse and the data mart, institutional research is becoming increasingly 
involved as a seconds^ data custodian. This is a natural extension of the institutional research 
role, evolving out of our responsibility to obtain, maintain, and retain reliable and valid data for 
key surveys such as the IPEDS data set. By obtaining and combining data from various 
operational sources at the institution, typically creating a census-type database, institutional 
research can demonstrate appropriate activities for other custodians at the institution. It acts as a 
steward for the data, not an owner of the data. It must help establish, audit, and share the 
definitions and descriptions of the data. It must balance access with security and must be 
concerned with the management of data. What items should be measured and on what cycle? 

What is a good enough way to collect, archive, and make the data available for longitudinal 
studies? A final issue concents how to capture and store data from external sources. The 
institutional research office frequently is the custodian for data such as in the Delaware Cost and 
Productivity data and the OSU faculty salary data. This responsibility includes documenting and 
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distributing the data It also includes being a conduit for data such as the IPEDS data that are 
made available on CD and the World Wide Web. 

The Broker. The Broker acts to restructure the data into information. This frequently 
involves interpreting the data within the cornea of the situation where they were collected. 
Statistics and other tools are valuable in reducing the amount of data, thus putting i, into a more 
usable form. Success is heavily dependent on knowledge of various technical skills such as the 
use of computers, software, networks, spreadsheet techniques, and quantitative tools as statistics, 
operations research, and econometrics. Also needed are qualitative analytical skills that suppori 
case study, situation assessment, and content analysis. Such skills must be integrated into the 
organization at the strategic, managerial, and operational levels as the institutional researcher 
moves to better understand the relevance of obsetvable events and use .ha, understanding to 
format or structure the information resulting from restructuring and analyzing the data. 

The Manager. To increase the usefulness of information at the strategic level, the 
institutional researcher needs to anticipate future uses of information in the institution as well as 
correctly defining its current situation. This requires a keen understanding of the ins, to, ion’s 
decision-making process being implemented by its managers, and an initial use of data to focus 
alternatives for those users. Increasing the organizational intelligence of the information users 
requires development of the political skills tha, interpret the customer’s situation. In addition, 
the institutional researcher must be able to assess the value of information on the manager's 
success or failure to deploy the information a, the point where i, makes its greatest contribution 
to the decision-making process. This requires an individual who can ac, no, only as a Manager 
who interfaces with others a, the boundary of the office while managing the office by 
simultaneously structuring institutional research projects to make them manageable. In other 
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words, the capable Institutional Researcher must be a qualified manager who has access to and 

knowledge of proper tools, who can develop appropriate policies, and who has the intetpersonal 
and organizational skills to involve the required staff in projects. 

Do - The Five Functions of Institutional Research 



Understanding the functions of institutional research begins with an understanding of the 
customers, including those who make decisions at the operational, managerial, and strategic 
levels. Data needs at the different levels are significantly different in tenns of timing, scope, and 
detail. For example, decisions at the strategic level tend to be long tern, and concerned with the 
future direction of the institution while those at the managerial level are more likely concerned 
with how the institution positions itself relative to other like institutions over the next two years. 
Decisions at the operational level are more likely to focus on the day-to-day operations of the 



institution. 

Institutional research needs to manage to give a sufficient base for reporting, research, 
and relational activities that address all levels of decision-making. This necessitates that 
institutional research be involved with, and gain an understanding of, functions that prov.de the 
raw materials for analysis and research, including information technology and data management. 



If needs of different customers are properly identified and the quality of data maintained, 

analyses can produce descriptive, comparative, or projective results that satisfy the needs of all 
decision-makers. 

Processes and projects should also be designed in a manner that represents an 
understanding of where the customer spends time and resources, and of how the customer values 
the various activities. Skills for conducting institutional research thus include technical 
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abilities/skills, understanding of higher education functions and processes, and an awareness of 
the institution (Terenzini, 1993 ). 

The strategic application of the skills and abilities that add value to the data, information 
and organizational intelligence provided by institutional research can be described by the 
following five functions. 

Situation Modeling. This starts the problem definition for decision-making. What is the 
situation? What are the primary characteristics of the problem? How do we know it is a 
problem? What is it that the institution really needs to know? The definition of the problem 
comes with the exploration and clarification of the situation and then the crystallization of what 
will be done to deal with the situation. If the situation of concern requires that a decision be 
made at a known point, organization of the project must include when the decision needs to be 
made. In cases where a sequence of events lead to a decision point, the timeline needs to be 
considered because good data and effective institutional research take time. 



Following the situation assessment, the appropriate measures must be identified and 



incorporated into a parsimonious model. While the measures may be framed as performance 
indicators or critical success factors, more typically they are framed as items that can help 
understand “what-causes-what” and to anticipate the outcomes resulting from a decision. If the 



need is to develop a model to monitor or anticipate specific events, a good starting point is to 
identify six or seven key components of the situation and selecting two or three measures 



developed from these components. The resulting measures can be organized into a conceptual 
model or analysis of manageable size that helps make decisions. 

Data Acquisition. What data are available? Are available data sufficient to provide the 
needed information? While some data are available for any problem that must be addressed, 
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institutional research must determine if, he data are sufficient to answer the problem Key data 
issues include their timeliness and reliability. 

A second concern tha, must be addressed is the availability of resources for securing the 
data. In general, a well developed understanding of the situation helps the institutional research 
function anticipate needed dam, usually producing better data at reduced costs. If the data are 
external to the college, obtaining them may involve the identification of resources required to 
obtain the data. In other cases, substantial monetary resources may no, be required and 
secondary databases may be available, such as the NCES dam bases. Often there will be trade- 
offs of time versus comprehensiveness that must be factored into the dam collection process. 
Often, when using secondary databases, the political issues concerning perceived believability, 
value, and appropriateness of the source must be addressed. If the dam are for interna, uses and 
no, to be made public, then agreement on masking and the public release of information need to 
be discussed and agreed upon. If collecting prima^ dam to establish an internal database, then 
institutional research needs to establish its own ability to manage the resulting database. 

Data Conversion. Wha, do the dam mean? Converting the dam to information involves 
restructuring die dam into the parameters defined by the problem and analyzing ,„e dam to focus 
them on the situation. The key to addressing understandability is to recognize that detailed data 
must be restmctured in ways tha, keep the important detail while simplifying a„ d su mmarizing 
“noise” in the dam. To accomplish this, the institutional researcher should consider use of 
multiple dam sources where possible since the same indicator coming from multiple sources is 
often more compelling. Also consider the perspective of those using the information. The 
analysis should include some conceptual components, some empirical components, and some 
experiential components, the three basic types of evidence that various individuals find 
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compelling. The process of successfully converting data to useful information must integrate 
data from multiple sources, make sure that there are consistent interpretations, and determine 
when sufficient data are available for making interpretations with adequate confidence. 

Information Reporting. To whom do the results need to be delivered? How should the 
findings be interpreted? Interpretation and integration of information into the context of the 
situation occurs when the results are delivered to the manager. Putting the information into the 
context of the manager increases the level of organizational intelligence about the situation. 

Access to the data and information must be provided in various forms. The advanced 
customer may want to conduct additional analysis. The administrator may want to check 
indicators to see if reality is consistent with his/her belief. The key focus for institutional 
research is to appropriately generalize the results from various analyses to specific situations. 

The conditions under which the data (supporting the information reported) were collected must 
be reflected. Understanding changes in primary influences in the environment, which might 
affect the data, will assist the manager in interpreting the implications of the information for 
his/her situation. Additional information that should be provided includes estimates of the 
confidence that can be placed on the findings and conclusions about causality if appropriate. 
Discussing the desirability of the specific outcomes which most likely can be achieved from 
specific decisions may or may not be included depending on what the manager or decision- 
maker needs. 

Information Use. What actions do the data, information, and interpretation support? 
What are the next steps? After the interpretation is provided to the manager, there is an increase 
in the organizational intelligence related to what is known about the situation. In other words, 
the user's knowledge base has been expanded. However, the reduction of uncertainty further 
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depends on the timeliness, sufficiency, and relevance of the information being provided. The 
information must be available before a decision is made to have a value. If the results are 
provided after the decision, it will be a waste of the customer’s time. The sufficiency of the 
conclusions deteimines whether the customer is made aware of all key issues affecting the 
situation. The relevance of the conclusions determines whether the information focuses the 
manager on key issues without including a large number of suiplus facts that have little bearing 
on the situation. When the information adequately increases the intelligence of the customer, 
appropriate actions will more likely be taken and the focus of the custodian broker, and manager 
will shift to the next issue affecting the situation. Beliefs about the situation are changed and the 
next set of questions will seek a refined set of definitions and information. 

Check - Add Value by Identifying the Barriers to Effective Institutional Research 

Checking is part of the control and evaluation process required to ensure that a functional 
area achieves its goals. Proper implementation of the necessary processes requires an 
understanding of the barriers to effective institutional research (Delaney, 1997; Knight, 
Coperthwaite, & Moore, 1995). If ignored, barriers limit the effectiveness of the processes 
presented in the framework and may in some instances represent a failure of the institution to 
learn. Improvement in the institutional research function results when the barriers to learning are 
removed and a healthy learning organization created or restored (Senge, 1990). Use of effective 
evaluation and control tools to check or monitor the organization are critical in that such tools 

not only enable the identification of barriers but also assist in efforts to demonstrate the value 
added by institutional research activities. 

If institutional research can identify and resolve the factors that create barriers to 
effectiveness, then it will be able to better define its core competencies to its customers and to 
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secure its position within the institution. Advantages of identifying barriers to success were 
identified by Sun Tzu in the 6h Century B. C. First, identifying barriers help the organization 
determine the appropriate response. “There are some roads not to follow; some troops not to 
strike; some cities not to assault; and some ground which should not be contested” (Sun Tzu, 
1971). There are numerous reasons for walking away from some situations, but we can’t know 
which situations we should avoid until we know what the reasons. We have the choice of how 
we focus our resources but we need to know the barriers related to the specific options we have 
and the pros and cons of making a specific choice. 

Second, identifying barriers helps avoid protracted conflict. “When your weapons are 
dulled and ardor damped, your strength exhausted and treasure spent, neighboring rulers will 
take advantage of your distress to act” (Sun Tzu, 1971). Dealing with barriers allows institutional 
researchers to focus their resources on specific issues. If efforts are not made to identify and 
then deal with barriers, existing barriers will continue to fester and will limit the success of your 
operation. Once your energies are dampened by the continuing conflict, the organization will 
sense less need for your services. Institutional research must identify the barriers, deal with 
them, and avoid continuing, nonproductive struggles. 

Third, identifying barriers gives you the advantage. “Generally, he who first occupies the 
field of battle and awaits his enemy is at ease; he who comes later to the scene and rushes into 
the fight is weary.” By identifying the barriers, institutional research can act to eliminate the 
causes of the barriers. If you are able to establish a process for dealing with the problem, then 
with organizational support, you will be able to bring more useful information to bear on the 
situation. You will also be able to influence the group set to work on the problem and will 
possibly be in a better position to develop a strategy for eliminating the barrier. 
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Our research and experience of over the past 30 years has ted to identification and 
ling of five major types of barriers to effective institutional research (Table 1). The 
categories of barters have evolved ou, of reviews of the literature, from personal experiences, 
and from institutional researchers attending various workshops conducted to increase knowledge 
on how to improve the effectiveness of the institutional research function. The firs, barter 
concerns the problem of determining w ha, is imporiant. Workshop participants indicate that 
problems arise concerning both forecasting the need for specific types of information and 
making administrators aware of the emerging issues. A major concern among the researchers is 
their lack of involvement in the decision process itself There is a lack of a consistently, 

communicated concerns from managers and this instability prevents us from developing the data 
structure needed to deal with problems as they emerge. 

The second barter concerns the failure of institutions to prcperly manage the collection 
of data. When data are obtained, there are often questions about how and when the data were 

obtained. Data are no, audited. Accessibility to appropriate data also appe^s to be a major 
problem at many institutions. 

The third barcier involves problems associated with the restructuring and analysis of data. 

Sometimes these problems are due to the unavailability of appropriate software This can be 

further complicated by the way data are structured and by the choice of methodologies for 

integrating intermediate results. In general, the broker looses touch with the information that can 
be best obtained from the data. 

The fourth barter concerns agendas within the mstitutional research function itself that 
interfere with effective delivery of information. I, ranges from the predisposition of offices to 
engage solely in routine reporting, to the perception tha, the purchase of new software and new 
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hardware can solve problems. This barrier tends to distract the institutional research from the 
tasks of ensuring that reports are understandable and contain summary information that enable 
the customer or end user to make recommendations. The information is not properly generalized 
to the situation. The organizational intelligence for dealing with the problems is not increased. 

The fifth, and perhaps the most dangerous, barrier to effective institutional research is 
political in nature. Lack of political skills by the institutional researcher can limit his/her ability 
to persuade the decision-makers at the strategic level that current studies are relevant and can 
effectively be used to support decision-making. The danger lies in the use by the customer of 
older reports with which they are familiar to address current problems that may need new and 
innovative solutions. Another potential problem surfaces when current studies are not sufficient 
to meet needs of the customer or the manager simply chooses not to use the studies. While the 
resulting decisions may be creative, the long-range results are typically disastrous. 

Act — Overcoming Barriers to Effective Institutional Research 

Acting involves implementation of purposeful tactics that are designed to demonstrate the 
relevance, sufficiency, reliability, and timeliness of institutional research to critical concerns of 
the institution (Suslow, 1972). Methodologies must be identified that effectively communicate 
relevant information to decision-makers in a timely manner. Goals must be pursued that are 
neither mundane nor perfunctory and which enable the institution to act to bring about 
meaningful change where needed. Ultimately, institutional research must support organizational 
learning through the systematic and appropriate balance of involvement and detachment in active 
assessment of policies and processes and support of decision-making. 

Knowledge of the strategic use of data and information are skills that can help overcome 
barriers to effective institutional research. When clarifying beliefs and identifying measures. 
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those in the institutional research function must demo „ s , rate experiise in negotiating ^ 

Planning projects. I, is a, this point, that the customer is ready to address a problem, i,„ there is 
a problem to be solved. Institutional research needs to identify a project or pmcess that wil, 
address the problem and add value to the institution in the eyes ofthe institution's various 
management groups. There are three basic approaches to getiing staried. First, institutional 
research can identify and evaluate an existing program that is relevant to the pmblem bu, tha, is 
no. meeting its objectives. Second, institutional research can identify relevant programs tha, are 
operating as planned bu, dm, lack sufficient utility to wa™, continuation or to solve the 

problem currently at hand. Third, institutional research can identify emerging nwd s tha, are 
relevant to solving the problem (Suslow, 1972). 

Given that ongoing negotiation detennines the ultimate focus of the project, institutional 
research needs to understand wha, the customer wants to do with the results If there is a 
decision to be made, the negotiation process should be used to clarify wha, the decision is and a, 
wha, point ofthe decision process institutional research is being asked to help. There is a 
sequence in the decision process that includes the following steps: 

1 Identify the problem leading to a need to make a decision. 

2. Clarify the issues related to the situation, and focus on the problem. 

3. Identify feasible alternatives, one of which is to avoid making a decision a, this time. 

4. Select the criteria for selecting the preferred alternatives. 

5. Suppon the decision tha, has been made and lobby for suppori ofthe position taken. 

6. Induce action in others to implement change. 

7. Measure and evaluate outcomes associated with the change. (Simon, 1983) 
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The customer may be seeking assistance in some or ail of the above steps. Once the 
purpose of the project has been clarified and the stage of needed support established, the 
framework within which the project should be conducted should be identified and refined. This 
requires that the institutional researcher be knowledgeable of the key processes of the institution 
to include what is known in the literature and what is currently being practiced at similar or 

comparable institutions. Participation in AIR and other relevant professions is also an obvious 
advantage. 

The requirements for the Act stage find their roots in the standards for good research, 
many of which have been part of the discussions of the Plan, Do, and Check activities. We have 
identified four standards that are especially important to the practice of effective institutional 
research. First, acting to add value to the institution requires that the institutional researcher be 
knowledgeable of the three basic characteristics of reliability - stability, objectivity, and 
consistency. Institutions that have implemented effective database management systems are in 
the best position to support institutional research with reliable data. Where such systems are not 
effectively managed, institutional researchers must work to establish minimum standards for 
dictionaries, data audit procedures, and policies for users of available data. 



Second, acting to add value requires the identification of a useful model for 
communicating the processes used by institutional research. For example, we use a five-step 
circular model to explain the processes used to conduct institutional research and to link this 



process to data needs (Figure 1). The concepts underlying the model are used in the negotiation 
process to facilitate communication and to explain a sequential process leading to 
recommendations. It contributes to the customer’s understanding of the implications associated 
with proper identification of the problem and facilitates the institutional researcher’s efforts to 
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gain the trust and confidence of the customer. It clearly demonstrates the need for cooperation 
among the Custodian, the Broker, and the Manager if the results are to be a high quality. 

Third, acting to add value requires management of the analysis of data, e.g., information 
management, in a way that reduces rather than raises barriers to effective communication 
between institutional research and the customer. Hackman (1983) suggests that analyses should 
be kept at a reasonable level. First, she advises that you should chunk your data wisely. This 
requires that those in the institutional research function identify a primary focus for the analysis 
and constrain the variables to the attention span of the manager. Second, she suggests that you 
augment humans with models. Managers like data, ideas, and human experiences and you 
should try to include facts from these three components. Third, she suggests that negative 
evidence and new hypotheses are okay. Hypotheses are important to effective research when the 
institution is working at the frontier of its knowledge base and where research efforts cannot 
possibly answer all the questions. 

Fourth, acting to add value requires effecting communication and reporting. Use 
reporting procedures that effectively answer the question in the language of the customer and 
without burdening the customer. In other words, if a graph can explain a relationship, do not 
present a regression analysis. If the customer prefers data tables, do not provide three- 
dimensional graphs. Be consistent in generating reports so that the customer can become 
familiar with the structure of information you present. This is especially important when the 
same report is generated on a periodic basis, e g., enrollment projections done on an annual basis, 
salary studies on peer groups. 
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CONCLUSIONS 

Over the past decade, we have seen strategic planning give way to strategic 
management. We have seen large mainframe computers make way for distributed desktop client- 
servers. Finally, we have seen operating systems give way to decision-support systems and 
executive information systems. The response of institutional researchers to these changes has 
been to define ourselves as knowledge workers and to position institutional research as a value- 
added function within the institution. This identification of our role is clarification of the 
profession of adding value to information as part of the techno structure. It is a role that was seen 
by early members of our profession but which is easy to confuse with the rapid changes in our 
institutions and in our tools. 

We continue to face the challenge posed by Sheehan over a decade ago when he asked, 
“Will the OIR disappear because central decision-makers will have their personal, mechanized 
decision support capability and no longer need the institutional research intermediary?” (1983, 
p. 194) He argued that the intervention of human judgment can make a unique contribution to 
the conceptualization of problems, formulation of analytical discovery processes, presentation 
and evaluation of results, and interpretations of the implications. (Sheehan, 1977, p. 95) 

Time has demonstrated the accuracy of his arguments concerning the value of human 
intervention. Institutional research has been given the opportunity to define its profession as the 
intermediary that can add the value to the institution through human intervention. For 
institutional research to fulfill that role, we must develop and refine appropriate knowledge, 
skills, and abilities. We need to identify the appropriate methodology. The models and thoughts 
presented above are presented as one way to frame discussion of an effective institutional 
research methodology. Plan, Do, Check, and Act can help us manage our functions. The cycle of 




24 



EFFECTIVE INSTITUnONAL RESEARCH 



23 



information support can help us operate our activities. We have found that these concepts and 
methodology helps us answer the traditional question: “And what do you do in Institutional 
Research?” 
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TABLE 1 




barriers to effective institutional research 



BELIEF BULIMIA — 

Definition: The semi-random gorging and purging of concepts, criteria, ar 
based on random changes in opinion of what is important. 

1. Difficulty in anticipating important needs for information 

2. Lack of agreement on the importance of various issues. 

3 . Lack of involvement of IR personnel in the decision process. 

4. Lack of priorities assigned to specific tasks and activities. 

Lack of awareness a mong administrators concerning problems and em 

DATA dyslexia — 

Definition: The inability to recall or recognize the meaning of the data or \ 

came from and the confusion of the data elements with other elements for 
longer know the meaning. 

1. Lack of managed systems where institutional research can access need. 

2. Lack of access to needed data from the new computer systems. 

3. Late arrival of data for use in decision-making. 

4. Unfounded conclusions that problems can be solved by adding data 
U- Lack of match between data from one system and data from other svste 

[dimensional dementia — — svste 

Definition: Results are not interpretable due to irrational combinations of c 
forgotten 8 ^ °" ^ available > leavin S the meaning of the info, 

1 . Lack of up-to-date analytical tools. 

2. Lack of documentation on data, access, and how reports are developed. 

3. Lack of usable studies due to complexity of the data. 

4. Lack of understanding due to complexity of methodology. 

IL Lack of familiarity with the results of the studies. 

MYOPIC MEGALOMANIA 1 - 

The se | f - centered and shortsighted delivery of information base 
ot the deliverer and independently of user needs. 

1 . Lack of flexibility to do real IR due to routine reporting 

2. Lack of understanding of reports. ° 

| 3. Lack of bottom-line summaries for reports. 

4. Difficulty in developing appropriate recommendations based on reports 
— Constant purchases of new software and hardw are to solve problems 
CREATIVE CARCINOMA E ~ — 

Definition: A pattern of creating and using facts as needed by using the Firs 
where the facts continue to be quoted until they cause a festering sore 

1 . Insufficient report for meeting the needs of the customer. 

2. Lack of problem-solving orientation with no suggested courses of action 

3. Non-relevant studies due to lack of knowledge concerning their use. 
Limitations on quality of results due to campus politics. 

5 Lack of use of information by those in leadership positions 
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